The present invention relates generally to systems and methods for transmitting data. More specifically, the present invention relates to systems and methods for efficiently scheduling and multiplexing multiple compressed bitstreams including video data using bit rate information describing the bit rate of the video data. In addition, the present invention relates to systems and methods for producing bit rate information describing the bit rate of video data and transmitting bitstreams including compressed video data and the bit rate information.
Video services are provided by a wide array of video content suppliers. For example, residential digital video services may include digital television, video on demand, Internet video, etc.xe2x80x94each service having hundreds of programs. A program refers to one or more bitstreams that are used to represent the video content and associated audio content. A target receiver for the programs, such as a set-top box (STB) located in a residential home, receives video programs from a number of different video content suppliers via assorted transmission channels. Typically, the xe2x80x98last milexe2x80x99 of transmission between the video content suppliers and the target receiver is along the same transmission channel, requiring the channel to carry multiple video programs from the wide array of suppliersxe2x80x94and often simultaneously.
There are presently a variety of different communication channels for transmitting or transporting video data. For example, communication channels such as coaxial cable distribution networks, digital subscriber loop (DSL) access networks, ATM networks, satellite, terrestrial, or wireless digital transmission facilities are all well known. In fact, many standards have been developed for transmitting data on the communication channels. For the purposes herein, a channel is defined broadly as a connection facility to convey properly formatted digital information from one point to another. A channel includes some or all of the following elements: 1) physical devices that generate and receive the signals (modulator/demodulator); 2) medium that carries the actual signals; 3) mathematical schemes used to encode and decode the signals; 4) proper communication protocols used to establish, maintain and manage the connection created by the channel 5) storage systems used to store the signals such as magnetic tapes and optical disks. The concept of a channel includes but is not limited to a physical channel, but also logical connections established on top of different network protocols, such as xDSL, ATM, IP, wireless, HFC, coaxial cable, Ethernet, Token Ring, etc.
The channel is used to transport a bitstream, or a continuous sequence of binary bits used to digitally represent compressed video, audio and/or data. A bit rate is the number of bits per second that is required to transport the bitstream. A bit error rate is the statistical ratio between the number of bits in error due to transmission and the total number of bits transmitted. A channel capacity is the maximum bit rate at which a given channel can convey digital information with a bit error rate no more than a given value.
Since transmission of video data with existing communication channels is often excessive, compression is an approach that has been used to make digital video images more transportable. Digital video compression allows digitized video data to be represented in a much more efficient manner and makes it possible to transmit the compressed video data using a channel at a fraction of the bandwidth required to transmit the uncompressed video data. For example, a digitized video data having an uncompressed bit rate of roughly 120 million bits per second (Mbps) can be represented by a compressed bitstream having a bit rate of 4-6 Mbps. Compression represents in significant data savings which results in much more efficient use of channel bandwidth and storage media.
When the digital video is first compressed, the encoder assumes a particular bit rate profile, whether it is constant bit rate (CBR) or a variable bit rate (VBR). The word xe2x80x9cprofilexe2x80x9d refers to the fact that transport bit rate may not be constant, but variable under certain constraints, such as peak bit rate, average bit rate, minimum bit rate, etc. For example, a constant bit rate stream at 4 Mbps does not have the same bit rate profile as a variable bit rate stream at an average of 4 Mbps but has larger maximum bit rate and smaller minimum bit rate, respectively.
The VBR representation of compressed video data allows a video encoder to generate compressed bitstreams that, when decoded, produce consistent video quality. However, as a result of the compression process, the number of bits required to represent the compressed data differs widely from picture to picture. The specific VBR characteristics of the compressed bitstream depends on the complexity of the video image, amount of motion in the video sequence, as well as changes made in post-generation such as scene cuts, fades, wipes, picture-in-picture, etc. As channel capacities are often expressed as constant bit rates, the variable nature of VBR compressed bitstream often poses a problem for video transmission.
One potential consequence of exceeding channel capacity for a VBR compressed bitstream on a particular channel is compromised video quality. Commonly, if one or more bitstreams is too large to fit within a channel, video data may be dropped from the bitstream or simplified to allow transmission; thus sacrificing end user video quality. Due to the real-time nature of compressed video transmission, dropped packets are not re-transmitted. Also, it is important to point out that compressed bitstreams are usually generated by either real-time encoders or pre-compressed video server storage systems. Both are likely to be in a remote site, away from the network itself. This increases the difficulty in encoding the video signal with a resulting bit rate profile sensitive to the connection bandwidth available for a particular channel or target receiver.
To further reduce the excessive amount of video transmission, bitstreams are frequently combined for transmission within a channel to make digital video data more transportable. A multiplex is a scheme used to combine bitstream representations of multiple signals, such as audio, video, or data, into a single bitstream representation. A re-multiplex is a scheme used to combine multiple bitstream representations of multiplexed signals into a single bitstream representation.
One important benefit of the VBR compression is achieved through the so-called xe2x80x98statistical multiplexingxe2x80x99. Statistical multiplexing is an encoding and multiplexing process which takes advantage of the VBR nature of multiple compressed video signals. When a statistical multiplexer combines multiple bitstreams, an algorithm may be used to adapt the bit rate of each VBR video signal but the total bit rate of the output multiplex is kept at a constant value. Statistical multiplexing encompasses multiplexing architecture having a reverse message path from the multiplexer to the encoders. This is also often referred to closed-loop statistical multiplexing.
FIG. 1A illustrates a high level architecture for a conventional closed-loop statistical multiplexer (statmux) 10. The closed-loop statmux 10 has a closed-loop signal path 11 between a statmux rate controller 12 and program encoders 14 and 16. The signal path 11 provides the rate controller 12 with a global view of the bit rate requirements for each of the program encoders 14 and 16 and allows communication between the rate controller 12 and each encoder. The encoders 14 and 16 provide compressed video, audio and data bitstreams to a multiplexer 15, which schedules the compressed bitstreams to output a multiplexed compressed bitstream 18. Each of the encoders 14 and 16 does not have knowledge of the bandwidth requirements of data being encoded by the other encoder and hence relies on messages sent by the rate controller 12. Based on these messages received from the statmux rate controller 12, the program encoders 14 and 16 adjust their encoding bit rate. Since the closed-loop statmux 10 relies on prompt delivery of the messages between the statmux rate controller 12 and the encoders 14 and 16, the closed-loop statmux 10 usually requires co-location of all program encoders 14 and 16, the rate controller 12 and multiplexer 15.
Statistical re-multiplexing, also called open-loop statistical multiplexing, or statistical rate re-multiplexing, is a process which performs statistical multiplexing of signals already in compressed format. Thus, statistical re-multiplexing includes accepting multiple VBR bitstreams and outputting a single CBR bitstream that fits within an available channel. In applications such as video on demand, digital cable headend systems, and digital advertisement insertion systems, statistical re-multiplexing may improve the overall system efficiency, resulting in better bandwidth usage and reduced transmission cost.
A conventional open-loop statistical re-multiplexer (stat remux) architecture 20 is illustrated in FIG. 1B. The architecture 20 includes an open-loop statistical re-multiplexer 21 that accepts compressed digital bitstreams consisting of multiple video/audio/data programs from encoders 22 and 24. The benefit of the open-loop stat remux architecture 20 is that it does not require reverse signal paths to the program encoders 22 and 24.
Functionally, the statistical re-multiplexer 21 does not control the bit rate output of each of program encoders 22 and 24. Although closed-loop statistical re-multiplexing can be highly efficient in bandwidth sharing among multiple encoded video/audio programs, it is not well suited for bandwidth sharing of multiple program encoders distributed over a large geographic area, or when multiple program streams are encoded and stored at different times. Even if such a reverse signal path exists, it must have low delay, and the program encoders 22 and 24 must be able to receive and correctly interpret the messages. Correct interpretation of the messages is often prevented when program encoders 22 and 24 in different geographic locations are not produced by the same manufacturer and implement different signal interfaces. Thus, network devices transmitting multiple video bitstreams typically use the open-loop stat remux architecture 20.
Unfortunately, the open-loop stat remux architecture 20 relies on information solely contained in the pre-compressed bitstreams for re-encoding. This reliance poses some limitations. One limitation is that the stat remux 21 cannot obtain information on the video signals within each compressed bitstreams it receives without completely decoding the signal back to the spatial domain (baseband). When the stat remux 21 is configured within a network device such as a router or headend, this complete decoding increases complexity of the network device, slows transmission of the video data, and decreases transmission efficiency. Any of these may diminish end-user video quality.
Based on the foregoing, improved methods and systems for transmitting video data over transmission channels would be desirable.
The present invention overcomes the deficiencies and limitations of the prior art with systems and methods for scheduling and multiplexing compressed bitstreams including video data with assistance from bit rate information describing the bit rate of video data included in one or more of the bitstreams. Two examples of bit rate information include coding statistics and raw video data statistics. The bit rate information is typically generated before multiplexing. Knowing compression related bit rate information ahead of time can be used to improve the scheduling and multiplexing efficiency of compressed bitstreams including video data and thus improve transmission efficiency of compressed video data over communication channels at bit rates that comply with the available bandwidth of the channels. For example, knowledge of bit rate information allows a scheduler to xe2x80x98stealxe2x80x99 bandwidth from one compressed bitstream and use it to improve the picture quality of another bitstream within the same multiplex. In addition, using the bit rate information associated with many coded pictures, before the coded picture data enters the scheduler, allows the scheduler to adopt a xe2x80x98look-aheadxe2x80x99 policy that makes scheduling and transcoding decisions based not only on the data to be immediately transmitted, but also on data to be transmitted ahead in time. Video data transmission in this manner may be used to improve video service quality for an end-user and to improve bandwidth usage efficiency while maintaining the video quality.
In one aspect, the present invention relates to a network device for providing a compressed bitstream including video data. The network device comprises an encoder apparatus that receives video data and outputs the compressed bitstream including the video data. The network device also comprises a processor apparatus that outputs bit rate information describing the bit rate of the video data. The network device further comprises a network interface that transmits an output compressed bitstream comprising the compressed bitstream including the video data and transmits the bit rate information.
In another aspect, the present invention relates to a method for providing compressed video data. The method comprises encoding video data to provide a compressed bitstream including the compressed video data. The method also comprises obtaining bit rate information describing the bit rate of the video data. The method further comprises transmitting an output compressed bitstream comprising the video data and transmitting the bit rate information.
In yet another aspect, the present invention relates to a network device for transmitting a first compressed bitstream including first video data and a second compressed bitstream including second video data onto a network. The network device comprises an extractor apparatus that obtains bit rate information describing the bit rate of the first video data. The network device also comprises a scheduler coupled to the extractor apparatus, the scheduler configured to receive the first compressed bitstream and receive the second compressed bitstream, the scheduler configured to use the bit rate information describing the bit rate of the first video data to schedule the first compressed bitstream and the second compressed bitstream, the scheduler also configured to output a compressed bitstream including the first video data and the second video data.
In still another aspect, the present invention relates to a network device for transmitting a first compressed bitstream including first video data and a second compressed bitstream including second video data onto a network. The network device comprises a processor apparatus that parses the first compressed bitstream and obtains bit rate information describing the bit rate of the first video data. The network device also comprises a scheduler coupled to the extractor apparatus, the scheduler configured to receive the first compressed bitstream and receive the second compressed bitstream, the scheduler configured to use the bit rate information describing the bit rate of the first video data to schedule the first compressed bitstream and the second compressed bitstream, the scheduler configured to output a compressed bitstream including the first video data and the second video data.
In another aspect, the present invention relates to a method for transmitting first video data and second video data over a transmission channel. The method comprises receiving a first compressed bitstream including the first video data and a second compressed bitstream including the second video data. The method also comprises obtaining bit rate information from the first compressed bitstream, the bit rate information describing the bit rate of the first video data. The method further comprises scheduling the first compressed bitstream including the first video data and the second compressed bitstream including the second video data using the bit rate information to provide a compressed bitstream comprising the first video data and the second video data. The method additionally comprises transmitting the compressed bitstream comprising the first video data and the second video data over the transmission channel.
In yet another aspect, the present invention relates to a system for providing compressed video data over a transmission channel. The system comprises means for receiving a first compressed bitstream including first video data and a second compressed bitstream including second video data. The system also comprises means for obtaining bit rate information from the first compressed bitstream, the bit rate information describing the bit rate of the first video data. The system further comprises means for scheduling the first compressed bitstream including the first video data and the second compressed bitstream including the second video data using the bit rate information to provide a compressed bitstream comprising the first video data and the second video data. The system additionally comprises means for transmitting the compressed bitstream comprising the first video data and the second video data over the transmission channel.
In still another aspect, the present invention relates to a computer readable medium including instructions for providing compressed video data over a transmission channel. The instructions comprises instructions for receiving a first compressed bitstream including first video data and a second compressed bitstream including second video data. The instructions also comprises instructions for obtaining bit rate information from the first compressed bitstream, the bit rate information describing the bit rate of the first video data. The instructions further comprises instructions for scheduling the first compressed bitstream including the first video data and the second compressed bitstream including the second video data using the bit rate information to provide a compressed bitstream comprising the first video data and the second video data. The instructions additionally comprises instructions for transmitting the compressed bitstream comprising the first video data and the second video data over the transmission channel.
These and other features and advantages of the present invention will be described in the following description of the invention and associated Figures.